Robot teaching program correction method

ABSTRACT

An industrial-robot teaching program is prepared off-line without using a robot. Then, cameras serving as vision sensors are mounted on a wrist flange of the robot to obtain measured position data for a teaching point P1. The teaching point P1 is on a workpiece in a sensor coordinate system and is detected by the vision sensor. The position data for the teaching point in the teaching program is converted into position data in the sensor coordinate system. Next, an error of the robot wrist flange coordinate system is estimated in accordance with a difference between measured position data for the teaching point and converted position data in the teaching program. Then, the teaching point data included in the teaching program is corrected based on the estimated error of the robot wrist flange coordinate system. The vision sensor may also be attached to an end effector of the robot.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a method for correcting, using a visionsensor, an already-prepared robot teaching program, that is, a robotteaching program in which teaching point data has already beenspecified. More particularly, the present invention relates to ateaching program correction method for removing the deviation between ateaching point position specified by a teaching program and a positioncorresponding to the teaching point on an actual work object. Though themethod of the present invention can be applied to industrial robots ingeneral, the application to a welding robot for performing arc weldingor spot welding may be considered as a typical application.

2. Description of the Prior Art

To rationalize the preparation process of the robot teaching programdesigned for various work, including welding work to be carried out bymoving a robot with respect to a work object (hereinafter referred to asa workpiece), an off-line programming system has been employed. Data fordesigning a workpiece, data relating to the positional relation betweena robot and the workpiece, and data giving the relation betweencoordinate systems set for the robot (homogeneous transformation matrixdata) are used to prepare a teaching program on an off-line basis.Therefore, an actual robot position (including its posture; the sameapplies hereafter) under a reproductive operation may deviate from aposition specified or prearranged as a teaching point on an actualworkpiece due to various error factors.

More particularly, it commonly happens that the actual position ofrobot, corresponding to the position taught by the teaching program whenthe program is executed, does not always coincide with the positionspecified as an n-th position on the actual workpiece. The following areerror factors which are considered to cause positional deviation oftaught positions.

1. Errors of data relating to the positional relation between a robotand a workpiece: In other words, the positional relation between therobot and the workpiece according to the data is not realized;

2. Machining and assembling errors of a robot (e. g. link length error);

3. Errors due to deflection of a robot caused by its own weight or theweight of an end effector;

4. Machining and assembling errors of an end effector;

5. Errors due to deflection of an end effector

6. Shape errors of a workpiece.

It has been difficult to prepare an off-line program by previouslyevaluating accurately these errors. In particular, it has been almostimpossible to previously evaluate the above factors 4 to 6.

Thus, the deviation between a position to be realized corresponding to ateaching point under reproduction of a program and a teaching pointposition specified for an actual workpiece typically occurs when theprogram was prepared on an off-line basis. However, the occurrence ofsuch deviation is not necessarily limited to the case of an off-lineprogram but may also occur in any prepared program into which the aboveerror factors may come. For example, when workpiece manufacturing lotnumbers are changed, the above factor 6 works. When end effectors arechanged, the above factors 3 to 5 work. Therefore, the aforementioneddeviation from teaching point may occur in a teaching program which wasprepared into any optional system.

Conventionally, however, when the above-described problem is encounteredor expected to be encountered, it has been a usual practice that therobot equipped with an end effector is manually operated to let its toolpoint coincide with each point on a workpiece (a representativeworkpiece) which is actually taught to the robot (by the program).However, the above manual program correction by an operator requires alarge workload. In particular, when types of workpieces are diversifiedand thereby the number of programs to be prepared and used increases,the program correction causes the productivity of the entire robotoperation to greatly decrease.

SUMMARY OF THE INVENTION

An object of the present invention is to solve the above problems of theprior art by providing a robot teaching program correction methodcapable of efficiently executing program data correction in order toprevent the above teaching point deviation, which occurs in a preparedteaching program. Moreover, another object of the present invention isto improve the usefulness of a program prepared by an off-lineprogramming system by providing the above teaching program correctionmethod.

To achieve the above objects, the present invention includes the step ofputting a mark which can be identified by a vision sensor on a point ofa workpiece corresponding to at least one teaching point selected out ofthe teaching points provided with position data on a prepared teachingprogram, the step of measuring the position of the mark by the visionsensor mounted on a robot while keeping the robot at the selectedteaching point position, the step of determining necessary correctionamounts of the position data for all or some teaching points in theteaching program in accordance with the mark position measurementresult, and the step of executing the position data correctioncorresponding to the necessary correction amounts.

Since the present invention has the above constitution, teaching-datacorrection for compensating errors due to various factors, which arehard to be measured, included in a robot teaching program prepared by anoff-line programming system is executed according to a simple procedureby using a vision sensor. Therefore, when the teaching program correctedin accordance with the present invention is executed, a robot track isrealized, which is compensated for all or substantial portions of (1)the errors of the data showing the positional relation between a robotand a workpiece, (2) the machining and assembling errors of a robot, (3)the errors due to deflection of a robot caused by its own weight or theweight of an end effector, (4) the machining and assembling errors of anend effector, (5) the errors due to deflection of an end effector, and(6) the shape errors of a workpiece. As a result, it becomes possible toomit a teaching-program data correction work which has been performed bycomplicated manual operation requiring skill and also to improve thereliability of the program data accuracy.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a typical view showing the state of teaching point positionmeasurement when applying the correction method of the present inventionto a teaching program for welding work;

FIG. 2 is a block diagram of main portions showing the outline of asystem constitution used to execute the method of the present invention;and

FIG. 3 is a flow chart showing the outline of an operation/processingprocedure for executing the teaching program correction method of thepresent invention for the case shown in FIG. 1 using the system shown inFIG. 2.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

FIG. 1 is a typical perspective view showing the state of teaching pointposition measurement when applying the correction method of the presentinvention to a teaching program for welding. FIG. 2 is a block diagramof main portions showing the outline of a system constitution used toexecute the method of the present invention.

In FIG. 1, reference numeral 1 represents a workpiece for teaching pointposition measurement fixed in a work space by a fixing means notillustrated, which has a weld line 6. A workpiece is selected, as aworkpiece for teaching point position measurement, from a large numberof similar type workpieces with the same size each having a weld line atthe same position. Reference numeral 2 represents a robot body whosewrist flange portion is provided with a welding torch 3 and two cameras4 constituting a three-dimensional vision sensor.

Symbols P0 to P3 represent teaching points set along the weld line 6. Ateaching program prepared on a off-line basis is provided with teachingpoint data (generally including errors) showing the positions of theseteaching points.

In the case of the present embodiment, the teaching point P1 is selectedas a typical teaching point and the mark "+" is put on a portion of aworkpiece corresponding to the teaching point P1. Such a mark can be ofany shape as long as it can be recognized by a vision sensor. Thus, themark can be formed by writing with ink or by attaching a seal. Moreover,it is also possible to use a characteristic point peculiar to aworkpiece as a mark. In FIG. 1, a state is illustrated in which the mark"+" is picked up by the cameras 4 and 5 while the robot is positioned ata robot position specified as the position of the teaching point P1 by ateaching program. If program data is in a state not requiringcorrection, the position of the tool tip set for the robot mustprecisely coincide with the mark position. In general, however, the bothwill not coincide with each other due to various error factors. That is,the program data requires correction.

Referring to FIG. 2, the system comprises a robot body 2, cameras 4 and5 (two cameras), an image processor 10, a robot controller 20, a weldingtorch controller 30, and an off-line programming system 40.

The image processor 10 is provided with a central processing unit(hereinafter referred to as a CPU) 11. The CPU 11 is connected to aframe memory (image memory) 12, a control software memory 13 comprisinga ROM, a program memory 14 comprising a RAM or the like, a data memory15 comprising a nonvolatile RAM, a camera interface 16, an imageprocessor 17, and a communication interface 18 through a bus 19.

The camera interface 16 is connected to the cameras 4 and 5. Imagesignals output from the two cameras 4 and 5 are input in order when theCPU 11 specifies a connector number in the camera interface 16. Thecommunication interface 18 is connected to a communication interface 27on the side of robot controller 20, and signals representing commands ordata are transmitted or received through these interfaces.

Images picked up in the fields of view of the cameras 4 and 5 areconverted into shaded images according to gray scale and stored in theframe memory 12. The image processor 17 has a function for processing animage stored in the frame memory 12 in accordance with a commandoutputted from the CPU 11. The control software memory 13 stores acontrol program for the CPU 11 to control a vision sensor, a calibrationprogram for setting a sensor coordinate system by using a jig, an imageanalysis program for detecting a weld line position by using the imageprocessor 17, and a program for commanding transmission of measurementdata to a robot at a proper time.

It is also possible to connect a TV monitor for visually confirming animage picked up by the camera 30 or an image called out from the framememory 12 through a monitor interface (not shown).

On the other hand, the robot controller 20 is provided with a centralprocessing unit (CPU) 21. The CPU 21 is connected, through bus 28, to aROM 22 storing a control program, a RAM 23 used to temporarily storecalculation data, a memory 24 comprising a nonvolatile RAM for storingvarious set values such as teaching data and calibration jig data, ashaft controller 25 (including a servo circuit) for controlling eachshaft of the robot body 40, a teaching control panel 26 for manualoperation of a robot, coordinate system setting, position teaching,automatic operation (reproductive operation), and communication with animage processor for transferring a sensor start-up command, etc., andthe communication interface 27 connected to the communication interface18 on the side of image processor 10.

The communication interface 27 is also connected to the welding torchcontroller 30 for turning on/off the welding torch 3 (see FIG. 1) andcontrolling welding voltage and welding current, and is also connectedto the off-line programming system 40. The interface 27 also serves asan input/output unit for the controller 30 and the system 40.

Though the above system constitution and functions are basically thesame as a three-dimensional vision sensor of a conventional weldingrobot, it has the following features for putting the present inventioninto practice.

<1> A program for obtaining the position data (on the sensor coordinatesystem) of the teaching point mark shown in FIG. 1 and other necessarydata are stored in the control software memory 13 of the image processor10.

<2> A program for calculating a necessary correction amount forcorrecting the teaching point data included in a teaching program byusing the above obtained position data of the teaching point mark, aprogram for correcting the position data for each of the teaching pointsP0 to P3 in accordance with the above calculation result, and relatednecessary data are stored in the nonvolatile memory 24 of the robotcontroller 20.

The following is the description of the principle for obtaining anecessary correction amount of the teaching point data included in theteaching program.

Four cases are described below which are assumed by combining the casesin which the cameras 4 and 5 serving as vision sensors are set to thewrist flange of the robot instead of an end effector, the cameras areset to the wrist flange of the robot through the end effector, andmoreover a case in which the vision sensor is a three-dimensionalsensor, and a case where it is a two-dimensional sensor.

[1] When the cameras 4 and 5 serving as vision sensors are set to thewrist flange of the robot instead of the end effector (welding torch):

First, data showing the positional relation between the sensorcoordinate system, of the three- or two-dimensional vision sensor, andthe wrists flange coordinate system, showing the position of the wristflange of the robot, is obtained by a proper calibrating operation. Thehomogeneous transformation matrix (hereinafter referred to simply asmatrix) showing the relative positional relation of the sensorcoordinate system to the wrist flange coordinate system is assumed to beC. Though there are several calibration methods, their descriptions areomitted here because they are generally known.

Moreover, it is assumed that the tool tip position with respect to thewrist flange is already set. Furthermore, it is assumed that the matrixshowing the relative positional relation of the tool coordinate systemto the wrist flange coordinate system is T. Furthermore, it assumed thata matrix R represents a wrist flange position in terms of one workpiececoordinate system fixed in a work space which is calculated based on theteaching program data, the matrix showing an actual (true) wrist flangeposition is given as R', and a teaching point position is given as P.

[1--1] When the vision sensor system uses a three-dimensional visionsensor:

When it is assumed that a measured position on the sensor coordinatesystem, in the case where the vision sensor is able to measure thethree-dimensional position of a teaching point mark put on a workpiece,is V' and a teaching point position calculated with program data is V,the following expressions are obtained:

    RCV=P                                                      (1)

    R'CV'=P                                                    (2)

When it is assumed that the error of R' to R is only due to a parallelmovement component, the following expression (3) holds.

    R'=R+Er                                                    (3)

Where Er is a matrix expressed by the following expression (4). ##EQU1##

The following expression (5) holds in accordance with the aboveexpressions (1) to (4).

    (R+Er)CV'=RCV                                              (5)

When rearranging the expression (5), the following expression isobtained.

    ErCV'=RC(V-V')                                             (6)

Thus, the components e_(x), e_(y), and e_(z) of the errors of R' to R onthe workpiece coordinate system are obtained in the form of theexpression below. ##EQU2##

From the above, it is estimated that the error of the wrist flangecoordinate system of the robot is RC(V-V') shown by the expression (7).Therefore, it is possible to obtain a teaching program including theteaching point data in which the error in terms of the coordinate systemof the wrist flange is compensated by shifting, for each teaching point,teaching point data (on the coordinate system of workpiece) by -RC(V-V')for collection.

Thus, it is possible to improve the coincidence degree between the robottrack under reproduction of a program and the teaching line prearrangedon the workpiece by absorbing a considerable part of the error factorsexcluding those related to an end effector.

[1-2] When the vision sensor system uses a two-dimensional visionsensor:

When it is assumed that the measured position of a teaching point markput on a workpiece, expressed on the sensor coordinate system by thetwo-dimensional-position-measuring vision sensor, is V' and a teachingpoint position calculated based on program data, expressed on the sensorcoordinate system, is V, the following expressions are obtainedsimilarly to the case of three-dimensional vision sensor.

    RCV=P                                                      (8)

    R'CV'=P                                                    (9)

However, since V' is originally two-dimensional information, it is shownby the following expression (10) including a parameter "t". ##EQU3##

The parameter "t" represents a value corresponding to the distance fromthe origin to a teaching point on the sensor coordinate system.Therefore, the Z-coordinate value of V is obtained from the aboveexpression (8), "RCV=P" to use the value as the value of "t". The above"u'" and "v'" are values obtained from the two-dimensional vision sensorsystem. This results in the establishment of a three-dimensionalcoordinate system having the Z axis coinciding with the optical-axisdirection of a visual sense system as the sensor coordinate system forexpressing the information obtained from the vision sensor system. Thus,it becomes possible to treat V' as three-dimensional data. Therefore, itbecomes possible thereafter to determine the necessary correction amountof teaching point data by the same procedure as the case of thethree-dimensional vision sensor system.

[2] When a camera serving as a vision sensor is set to an end effector(welding torch):

It is assumed that the tool tip point position (including its attitude)with respect to the wrist flange is previously set. Moreover, it isassumed that the matrix representing the relative positional relation ofthe tool coordinate system with respect to the wrist flange coordinatesystem calculated in accordance with the data assumed by a program is T,and the matrix actually representing the relative positional relationbetween the both systems is T'.

Moreover, through a proper calibrating operation, data is obtained,which represents the positional relation between the sensor coordinatesystem of a three- or two-dimensional vision sensor and the toolcoordinate system. Furthermore, it is assumed that the matrixrepresenting the relative positional relation of the sensor coordinatesystem to the tool coordinate system is K. Though there are severalcalibration methods, their descriptions are omitted here because theyare generally known.

Furthermore, it is assumed that the matrix showing the wrist flangeposition viewed on one workpiece coordinate system fixed in a workspace, calculated in accordance with teaching program data, is given asR, the matrix showing an actual (true) wrist flange position is given asR', and a teaching point position is given as P.

[2-1] When the vision sensor system uses a three-dimensional visionsensor:

When it is assumed that a measured position on the sensor coordinatesystem in the case where the vision sensor is able to measure thethree-dimensional position of a teaching point mark put on a workpieceis V' and a teaching point position calculated from program data is V,the following expressions are obtained:

    RTKV=P                                                     (11)

    R'T'KV'=P                                                  (12)

The above expressions (11) and (12) are obtained by substituting RT andR'T' for R and R' and K for C in the expressions (1) and (2). Therefore,by calculating the following expression (13) instead of the expressionRC(V-V') for the calculation of a necessary correction amount in thecase of the above [1--1], a necessary correction amount in this case isobtained.

    RTK(V-V')                                                  (13)

[2-2] When the vision sensor system uses a two-dimensional visionsensor:

When performing calculation by replacing R and R' with RT and R'T' and Cwith K as in the case of the above [2-1], it is possible to bring backto the case of the above [1-2].

If it is possible to assume that the vision sensor is mounted in aposition where machining errors and deflection of an end effector itselfcan be reflected, the compensation effect of these errors can also beexpected. Moreover, even if the vision sensor is mounted in a positionwhere machining and assembling errors of the end effector or deformationerrors due to loads are not reflected (parallel simultaneous setting),the deflection of a robot arm due to the weight of the end effector isreflected on the result of mark position measurement. Therefore,function for compensating the error due to the deflection of the robotarm caused by the weight of the end effector is ensured.

The correction calculation corresponding to each of the previouslydescribed cases can be executed in a normal robot controller connectedwith the vision sensor; however, it is also possible to execute theabove corrective calculation by transferring the measurement resultobtained by using the vision sensor to an external unit (e.g. off-lineprogramming system) other than the robot controller.

The following is the description of the procedure and processing whenapplying the teaching program correction method of the present inventionto the case shown in FIG. 1 by using the system shown in FIG. 2.

First, a program prepared by the off-line programming system 40 istransferred to the robot controller 20 through the communicationinterface 27 to store the program in the nonvolatile memory 24 (stepS1). Then, the robot is moved to the position specified as the teachingpoint P1 on the program (step S2).

In this condition, vision sensors (cameras 4 and 5) are started to pickup the image of a mark set correspondingly to the teaching point P1 toobtain the mark position data V' on the sensor coordinate system (stepS3). This data is immediately transferred to the robot controller 10through both communication interfaces 18 and 27 and stored in thenonvolatile memory 24 (step S4).

A necessary correction amount of the teaching point position data on theprogram is calculated in accordance with the calculation methoddescribed above (step S5). Because the case in FIG. 1 assumed herecorresponds to the case [2-1], it is possible to obtain a necessarycorrection amount by calculating the expression (13). Furthermore, sincethe values of R, T, K, and V are already input to the robot as presentdata or set data, these data can be used for the calculation of theexpression (13).

After the necessary correction amount is calculated, a correctedteaching program can be obtained by subtracting the necessary correctionamounts from the position data of each of the teaching points P0 to P3(step S6).

Most of the error factors 1 to 6 described above are eliminated inaccordance with the program correction of this embodiment. That is, itis considered that errors of the robot wrist position include at leastsome of the following errors: 1. errors of the data showing thepositional relation between a robot and a workpiece, 2. machining andassembling errors of a robot, 3. errors due to deflection of a robotcaused by its own weight or the weight of an end effector, and 6. shapeerrors of a workpiece. Therefore, by compensating the errors of therobot wrist position, it is expected that these error factors arecompensated to a considerable extent. If it is possible to assume thatthe cameras 4 and 5 are set to a position where machining errors anddeflection of a welding torch (end effector) can be included, thecompensation effect of the errors can also be expected.

In the case of the above embodiment, one teaching point is selected as ameasuring object and the data for all teaching points including otherremaining teaching points are corrected in accordance with the positiondata for a mark put on the selected teaching point. However, there is notheoretical restriction in selecting a combination of a teaching pointto be selected as a measuring object with teaching points whose data areto be corrected in accordance with the selected teaching point. Ingeneral, it is possible to calculate a necessary correction amount foreach point to be corrected by selecting one or more teaching pointswhich can be considered to typically reflect a common error inaccordance with kinds of application or the environment in individualcases and measuring one by one the position of a mark put oncorresponding to each selected teaching point. It is a matter of coursethat correction can be omitted for a teaching point such as an air cutpoint which does not require a very high accuracy.

What is claimed is:
 1. A robot teaching program correction method forcontrolling a robot having a vision sensor comprising the stepsof:forming a mark which can be identified by the vision sensor on aportion of a workpiece corresponding to at least one teaching pointselected from teaching points in a teaching program, wherein each of theteaching points has corresponding position data; measuring a position ofthe formed mark by the vision sensor while maintaining the robot at theselected teaching point; obtaining correction amounts corresponding tothe position data for some teaching points in the teaching program inaccordance with the measured mark position; and correcting the positiondata in accordance with the correction amounts.
 2. A robot teachingprogram correction method for controlling a robot comprising the stepsof:(1) preparing an industrial-robot teaching program including aplurality of teaching points and corresponding program position dataoff-line without using the robot; (2) obtaining measured position dataof one teaching point on a workpiece in a sensor coordinate system bymounting a vision sensor to a wrist flange of the robot and detectingthe one teaching point by the vision sensor; (3) converting programposition data for the detected one teaching point in the teachingprogram into program position data in a sensor coordinate system inaccordance with a calibrated positional relation between the sensorcoordinate system and a robot wrist flange coordinate system; (4)estimating an error of the robot wrist flange coordinate system based ona difference between the measured position data for the one teachingpoint obtained in the above Item (2) and the program position dataobtained in the above Item (3); and (5) correcting program teachingpoint data corresponding to the teaching points included in the teachingprogram based on the estimated error of the robot wrist flangecoordinate system estimated in the above Item (4).
 3. A robot teachingprogram correction method comprising the steps of:(1) preparing anindustrial-robot teaching program including a plurality of teachingpoints and corresponding program position data off-line without usingthe robot; (2) obtaining measured position data for one teaching pointon a workpiece in a sensor coordinate system by mounting a vision sensorto an end effector of the robot and detecting the one teaching point bythe vision sensor; (3) converting program position data for the detectedone teaching point in the teaching program into program position data ina sensor coordinate system in accordance with a calibrated positionalrelation between the sensor coordinate system and a tool coordinatesystem; (4) estimating an error of the tool coordinate system based on adifference between the measured position data of the one teaching pointobtained in the above item (2) and the program position data obtained inthe above Item (3); and (5) correcting program teaching point datacorresponding to the teaching points included in the teaching programbased on the estimated error of the tool coordinate system estimated inthe above Item (4).
 4. The robot teaching program correction methodaccording to claim 2, wherein the detection of the one teaching point onthe workpiece using the vision sensor is performed by forming a markwhich can be recognized by the vision sensor on a portion of theworkpiece corresponding to one of the teaching points and detecting themark by the vision sensor.
 5. The robot teaching program correctionmethod according to claim 2, wherein the detection of the one teachingpoint on the workpiece by the vision sensor is performed by detecting acharacteristic point inherent to the shape of the workpiece whichcorresponds to the teaching point by the vision sensor.
 6. A method forcorrecting a robot teaching program which includes a plurality ofteaching points comprising the steps of:forming a mark on a portion of aworkpiece; identifying the formed mark by a robot vision sensor;selecting a teaching point from the plurality of teaching points in theteaching program corresponding to the identified mark; measuring aposition of the identified mark by the vision sensor; and correcting aportion of said plurality of teaching points by calculating a differencebetween the selected teaching point and the measured position of theidentified mark.